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AMENDMENTS TO THE CLAIMS 

1 . (Currently Amended): A system for receiving audio input comprising: 

a display for displaying electronic informatio n, wherein the electronic information 
includes a plurality of documents : 

an audio input receiving audio annotations in the form of audio clips; 

a processor for associating said received audio annotations with said displayed electronic 
information, wherein said processor further associates an author of each of said audio 
armotations with each of said respective audio annotations, each of said audio annotations being 
randomly accessible based on the author; 

a storage for storing the audio clips, wherein the storage of the audio clips includes 
properties that permit audio information to be associated with a visual , wherein the properties 
include position data indicating the location in the electronic information at which the author 
inserted each audio clip and time data indicating the time of recording of each audio clip during a 
session so that one audio clip is display e d in synchronism with another audio clip ; and 

a navigation history feature for recording all document navigations indexed by time so 
that both the audio clip recorded during the session and a sequence of document navigations can 
be played back simultaneously: 

wherein said audio input receives verbally delimited kejwords and converts said verbally 
delimited keywords into search queries, and 

wherein the processor allows dynamically accessing the audio clips based on the search 
queries. 

2. (Canceled) 

3. (Canceled) 

4. (Previously Presented): The system according to claim 1 , wherein the storage stores 
each of said audio annotations with the associated author. 
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5. (Previously Presented): The system according to claim 1, further comprising: 
an input receiving a user's input, 

wherein said processor starts recording an audio annotation from said audio input in 
response to said user's input. 

6. (Previously Presented): The system according to claim 1, wherein said processor 
includes a voice activated recording system for recording said audio annotation. 

7. (Previously Presented): The system according to claim 6, wherein said voice activated 
recording system records when said audio annotation exceeds a predetermined threshold. 

8. (Previously Presented): The system according to claim 6, wherein said voice activated 
recording system records when a known user's voice is detected in said audio annotation. 

9. (Previously Presented): The system according to claim 1, wherein said processor 
controls said display to indicate that an audio annotation is associated with said displayed 
electronic information. 

10. (Currently Amended): A system for playing audio content, said system comprising: 

a display for displaying electronic information , wherein the electronic information 
includes a plurality of documents ; 

a storage for storing audio annotations in the form of audio clips, each of said audio 
annotations including an author of said audio annotation and being associated with said 
displayed electronic information, each of said audio annotations being randomly addressable 
based on the author; wherein said storage includes properties that permit audio information to be 
associated with a visual , wherein the properties include position data indicating the location in 
the electronic information at which the author inserted each audio clip and time data indicating 
the time of recording of each audio clip during a sessio n so that one audio clip is display e d in 
synchronism with anoth e r audio clip ; and 
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a navigation history feature for recording all document navigations indexed by time so 
that the system can simultaneously play back, not only the audio clip recorded during the 
session, but also a sequence of document navigations; 

an audio input for receiving verbally delimited keywords and converting said verbally 
delimited keywords into search queries; 

an output for outputting at least some of said audio annotations responsive to navigation 
of said displayed electronic information; and 

a processor for controlling said display, said storage and said output, 

wherein the processor allows dynamically accessing the audio clips based on the search 
queries. 

11. (Canceled) 

12. (Canceled) 

13. (Previously Presented): The system according to claim 10, wherein said storage is a 
database. 

14. (Previously Presented): The system according to claim 10, fiirther comprising: 
an input for receiving a user's input, 

wherein said output outputs at least some of said audio annotations in response to 
receiving said user's input. 

15. (Previously Presented): The system according to claim 10, fiirther comprising: 
an input for receiving a user's input of an author, 

wherein said processor searches authors of said stored audio annotations in response to 
said user's input. 
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16. (Original): The system according to claim 15, wherein the output of said processor is 
sent to said display to display an indication of the search results. 

17. (Previously Presented): The system according to claim 15, wherein the output of said 
controller is sent to the output for playing an audio annotation with an author matching the 
search results. 

18. (Previously Presented): The system according to claim 10, wherein said processor 
retrieves all audio annotations associated with said electronic information when said electronic 
information is accessed. 

19. (Previously Presented): The system according to claim 10, wherein said processor 
outputs selected audio annotations to be played through said output when a page of said 
electronic information is displayed. 

20. (Previously Presented): The system according to claim 19, wherein said processor 
automatically plays said selected audio annotations when said page is displayed. 

21. (Previously Presented): The system according to claim 10, further comprising: 
a communication link to transmit said audio annotations. 

22. (Previously Presented): The system according to claim 21, further comprising: 

a network connected to said communication link for receiving said audio annotations, 
said network being accessible by other users. 

23. (Previously Presented): The system according to claim 21, further comprising: 

a receiving device of another user for receiving said audio annotations, said receiving 
device receiving said audio armotations through one of a wired or wireless interface. 
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24. (Previously Presented): The system according to claim 22, wherein said network 
further processes said audio annotations. 

25. (Previously Presented): The system according to claim 22, wherein said network 
includes a database for storing said audio aimotations. 

26. (Previously Presented): The system according to claim 22, wherein said network 
receives audio annotations without receiving said electronic information associated with said 
audio annotations. 

27. (Currently Amended): A user interface for displaying electronic information to a user^ 
the electronic information including a pluralitv of documents, the user interface comprising: 

a first display portion for displaying a portion of a document; and 

a second display portion for displaying a graphical indication that said document includes 
an audio aimotation associated with said displayed portion of said document, 

wherein said audio annotation is in the form of audio clips and wherein the audio 
annotation is associated with an author of said audio annotation, said audio clips are stored in a 
storage which includes properties that permit audio information to be associated with a visual^ 
wherein the properties include position data indicating the location in the electronic information 
at which the author inserted each audio clip and time data indicating the time of recording of 
each audio clip during a session so that on e audio clip is display e d in synchronism with another 
audio clip ; 

a navigation history feature for recording all document navigations indexed by time so 
that the svstem can simultaneously play back, not onlv the audio clip recorded during the 
session, but also a sequence of document navigations; 

an audio input for receiving verbally delimited keywords and converting said verbally 
delimited keywords into search queries; 

wherein the user interface is controlled by a processor which allows dynamically 
accessing the audio clips based on the search queries. 
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28. (Original): The user interface according to claim 27, further comprising: 
a third display portion for displaying a non-audio annotation. 

29. (Original): The user interface according to claim 27, further comprising: 

a third display portion for displaying an indication that said audio annotation is being 
recorded or played back. 

30. (Original): The user interface according to claim 27, further comprising: 

a third display portion for displaying one of a document tape or a master tape. 

31. (Original): The user interface according to claim 27, further comprising: 

a third display portion for receiving a user input of a property or properties of said audio 
annotation. 

32. (Original): The user interface according to claim 27, wherein said audio annotation is 
recordable by said user. 

33. (Currently Amended): A process for recording an audio annotation comprising the 
steps of: 

displajdng electronic informatio n, wherein the electronic information includes a plurality 
of documents ; 

receiving a user input; 

recording an audio armotation in the form of audio clips in response to said user input; 

associating an author of the audio annotation with the audio annotation; 

storing said audio clips in a storage which includes properties that permit audio 
information to be associated with a visual wherein the properties include position data indicating 
the location in the electronic information at which the author inserted each audio clip and time 
data indicating the time of recording of each audio clip during a sessio n so that one audio clip is 
displayed in synchronism with another audio clip ; and 
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recording all document navigations indexed by time so that both the audio clip recorded 
during the session and a sequence of document navigations can be played back simultaneously; 

associating said audio annotation with properties including a displayed portion of said 
electronic information; 

receiving verbally delimited ke3words and converting said verbally delimited keywords 
into search queries; 

dynamically accessing the audio clips based on the search queries. 

34. (Original): The process according to claim 33, further comprising the step of: 
storing said audio annotation prior to the association of said audio armotation with said 

displayed portion. 

35. (Original): The process according to claim 33, ftirther comprising the step of: 
storing said audio annotation after the association of said audio annotation with said 

displayed portion. 

36. (Original): The process according to claim 33, wherein said recording step records all 
ambient sounds. 

37. (Original): The process according to claim 33, wherein said recording step records 
only sounds above a predetermined threshold. 

38. (Original): The process according to claim 37, wherein said recording step records 
only a specific user's voice. 

39. (Original): The process according to claim 33, further comprising the step of: 
associating additional properties with said audio annotation at the start of recording of 

said audio annotation. 
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40. (Original): The process according to claim 33, wherein one of said properties is a file 
position or document position of an item on said displayed portion of said electronic information. 

41. (Original): The process according to claim 33, wherein one of said properties is a start 
identification of said displayed portion of said electronic information. 

42. (Canceled) 

43. (Currently Amended): A process for plajdng audio annotations comprising the steps 

of: 

displajdng a portion of electronic informatio n, wherein the electronic information 
includes a plurality of documents ; 
receiving a user input; 

retrieving audio aimotations in the form of audio clips firom a storage which includes 
properties that permit audio information to be associated with a visual , wherein the properties 
include position data indicating the location in the electronic information at which the author 
inserted each audio clip and time data indicating the time of recording of each audio clip during a 
sessio n so that on e audio clip is displayed in synchronism with another audio clip , each of said 
audio annotations being associated with an author of said audio aimotation; 

recording all document navigations indexed by time so that both the audio clip recorded 
during the session and a sequence of document navigations can be played back simultaneously; 

receiving verbally delimited keywords and converting said verbally delimited keywords 
into search queries; 

wherein said retrieving includes dynamically accessing the audio clips based on the 
search queries; 

assembling said audio clips into an audio stream; and 
playing said audio stream. 
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44. (Original): The process according to claim 43, further comprising the step of: 
waiting for a second user input prior to playing said audio stream. 

45. (Previously Presented): The process according to claim 43, further comprising the 

step of: 

playing said audio stream once said audio stream is assembled. 

46. (Original): The process according to claim 43, wherein said user input is a text query. 

47. (Original): The process according to claim 43, wherein said user input is a voice 

query. 

48. (Original): The process according to claim 43, further comprising the steps of: 
altering the display of said portion to match a currently playing aimotation in said audio 

stream. 

49. (Previously Presented): The process according to claim 48, wherein said altering step 
includes the steps of: 

comparing the length of said currently playing annotation with starting identifications of 
displayable portions of said electronic information; and 

displaying the portion of said electronic information supporting the greater length of said 
currently playing annotation. 

50. (Currently Amended): A process for playing audio annotations comprising the steps 

of: 

navigating to a page; 

retrieving at least one audio annotation associated with a page or associated with an item 
on a page in the form of audio clips firom a storage which includes properties that permit audio 
information to be associated with a visual , wherein the properties include position data indicating 
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the location in the page at which the author inserted each audio clip and time data indicating the 
time of recording of each audio clip during a sessio n so that on e audio clip is displayed in 
synchronism with anoth e r audio clip ; 

recording all page navigations indexed by time so that both the audio clip recorded during 
the session and a sequence of page navigations can be played back simultaneously; 

receiving verbally delimited keywords and converting said verbally delimited keywords 
into search queries; 

wherein said retrieving includes dynamically accessing the audio clips based on the 
search queries; and 

playing said at least one audio annotation, 

wherein said at least one audio annotation is associated with a property identifying an 
author of said at least one audio annotation. 

51. (Original): The process according to claim 50, further comprising the step of: 
waiting for a user input prior to playing said audio annotation. 

52. (Original): The process according to claim 50, wherein said item on said page 
includes at least one of embedded notes, inked notes, highlights and underlining. 

53. (Original): The process according to claim 50, wherein said at least one audio 
annotation was previously retrieved and said retrieving step includes indexing said previously 
retrieved at least one audio annotation. 

54. (Original): The process according to claim 50, wherein said at least one audio 
annotation is the result of a newly executed query. 

55. (Currently Amended): A computer readable medium having a data structure stored 
thereon, said data structure comprising: 

a document; 
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a link object; and 

audio content with at least one property, 

wherein said link object references said document and references said audio content, 

wherein said audio content includes at least one audio annotation in the form of audio 
clips, wherein the audio clips are stored in a storage which includes properties that permit audio 
information to be associated with a visual , wherein the properties include position data indicating 
the location in the document at which the author inserted each audio clip and time data indicating 
the time of recording of each audio clip during a sessio n so that on e audio clip is display e d in 
synchronism with anoth e r audio clip ; 

information for recording all document navigations indexed by time so that both the 
audio clip recorded during the session and a sequence of document navigations can be played 
back simultaneously; 

information for receiving verbally delimited keywords; 

information for converting said verbally delimited keywords into search queries; and 
information for dynamically accessing the audio clips based on the search queries; 
wherein at least one property identifies an author of the at least one audio aimotation. 

56. (Previously Presented): The data structure according to claim 55, wherein at least one 
property relates to the time said audio content started recording. 

57. (Previously Presented): The data structure according to claim 55, wherein at least one 
property relates to the time said audio content stopped recording. 

58. (Previously Presented): The data structure according to claim 55, wherein at least one 
property relates to the length of recording of said audio content. 

59. (Canceled) 
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60. (Previously Presented): The data structure according to claim 55, wherein at least one 
property relates to a start ID. 

61. (Previously Presented): The data structure according to claim 55, wherein sat least 
one property relates to a stop ID. 

62. (Canceled) 

63. (Previously Presented): The data structure according to claim 55, wherein said audio 
clips are stored in a database. 

64. (Previously Presented): The data structure according to claim 55, wherein at least one 
property is one of a plurality of properties and said properties are in a marked up language form. 

65. (Original): The data structure according to claim 64, wherein said properties are in 

XML. 

66. (Original): The data structure according to claim 55, wherein said audio content is 
stored within a document. 

67. (Original): The data structure according to claim 55, wherein said audio content is 
stored apart from a document. 

68. (Previously Presented): The data structure according to claim 67, wherein said audio 
content is stored in a database with at least one property designating a position of a viewed 
document relating to said audio content. 
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69. (Previously Presented): The data structure according to claim 67, wherein said audio 
content is stored in a database and linked to a separate annotation document that stores a position 
of a viewed document relating to said audio content. 

70. (Currently Amended): A process for recording audio content comprising the steps of: 
navigating to a page of a document; 

recording said audio content; and 

associating properties with said audio content such that retrieval of said audio content 
positions said audio content after previously recorded audio content, 

wherein said audio content includes at least one audio aimotation in the form of audio 

clips, 

storing said audio clips in a storage which includes properties that permit audio 
information to be associated with a visual , wherein the properties include position data indicating 
the location in the page at which an author inserted each audio clip and time data indicating the 
time of recording of each audio clip during a session so that on e audio clip is display e d in 
synchronism with anoth e r audio clip ; and 

recording all page navigations indexed bv time so that both the audio clip recorded during 
the session and a sequence of page navigations can be played back simultaneously; 

receiving verbally delimited kejwords and converting said verbally delimited keywords 
into search queries; 

dynamically accessing the audio clips based on the search queries; 

wherein at least one property identifies aa-the author of the at least one audio annotation. 

71. (Cancelled) 

72. (Currently Amended): The process according to claim 74-70, wherein said audio 
content and said previously recorded audio content is ordered at least by said time prop e rt y data . 
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73. (Currently Amended): A process of searching audio clips inserted into an electronic 
information, the electronic information comprising a plurality of documents, wherein each of 
said audio clips includes an audio annotation associated with an author of the audio annotation, 
the process comprising the steps of: 

inputting search terms or properties; 

searching said audio clips for said search terms or properties from a storage which 
includes properties that permit audio information to be associated with a visual , wherein the 
properties include position data indicating the location in the electronic information at which the 
author inserted each audio annotation and time data indicating the time of recording of each 
audio annotation during a sessio n so that on e audio clip is display e d in synchronism with another 
audio clip ; and 

recording all document navigations indexed by time so that both the audio annotation 
recorded during the session and a sequence of document navigations can be played back 
simultaneously; 

ordering audio clips detected by said searching step for output; 

receiving verbally delimited keywords; 

converting said verbally delimited keywords into search queries, 
dynamically accessing the audio clips based on the search queries; 
wherein e ach of said audio clips includ e s an audio annotation associat e d with an author 
of the audio annotation, 

wherein said search terms or properties identify an-the author of the audio annotation. 

74. (Canceled) 

75. (Currently Amended): A process for recording audio information comprising the 
steps of: 

recording audio signals as a first file; 
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processing said file to extract audio clips inserted into an electronic information, the 
electronic information comprising a plurality of documents, wherein each of said audio clips 
includes an audio annotation associated with an author of the audio annotation ; and 

storing said audio clips in a storage which includes properties that permit audio 
information to be associated with a visual , wherein the properties include position data indicating 
the location in the electronic information at which the author inserted each audio annotation and 
time data indicating the time of recording of each audio annotation during a sessio n so that on e 
audio clip is displayed in synchronism with anoth e r audio clip , 

recording all document navigations indexed by time so that both the audio annotation 
recorded during the session and a sequence of document navigations can be played back 
simultaneously; 

receiving verbally delimited keywords; 

converting said verbally delimited keywords into search terms or properties; 

djaiamically accessing the audio clips based on the search queries; 

wherein said processing separates the content of said first file into audio clips based on 

events, 

wher e in e ach of said audio clips includes an audio annotation associated with an author 
of th e audio annotation, 

wherein said processing associates an author of the recorded audio signal with the audio 
annotation. 

76. (Original): The process for recording according to claim 75, 
wherein said audio signals include speech, and 

wherein said events comprise at least one of short pauses in said speech, a pause of a 
predetermined length, and a user navigating away from a displayed page. 

77. (Currently Amended): A process for associating audio notes and handwritten notes 
comprising the steps of: 

creating a handwritten note; 
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associating a time at which said handwritten note was created with said handwritten note; 
creating an audio note; 

associating a time at which said audio note was created with said audio note; and 

associating an author of said audio note with said audio note, wherein said audio notes 
are comprised of audio clips inserted into an electronic information, the electronic information 
comprising a plurality of documents, 

wherein, upon selection of said handwritten note, audio notes created at or near the time 
at which said handwritten note was created are located with said handwritten note, 

wherein locating said audio notes includes the steps of: 
receiving verbally delimited keywords; 

converting said verbally delimited keywords into search queries; 
dynamically accessing a database which includes the audio clips based on the 
search queries, and wherein the database includes properties that permit audio 
information to be associated with a visual , wherein the properties include position 
data indicating the location in the electronic information at which the author inserted each 
audio clip and time data indicating the time of recording of each audio clip during a 
session so that on e audio clip is displayed in synchronism with anoth e r audio clip and 

recording all document navigations indexed by time so that both the audio clip 
recorded during the session and a sequence of document navigations can be played back 
simultaneously . 

78. (Canceled) 

79. (Original): The process according to claim 77, wherein locating said audio notes 
includes the step of searching a table. 

80. (Original): The process according to claim 77, wherein locating said audio notes 
includes the step of searching a linked list. 
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81. (Cancelled) 

82. (Original): The process according to claim 77, further comprising the step of: playing 
said audio notes. 

83. (Currently Amended): A process for playing audio notes in the form of audio clips 
inserted into an electronic information by an author, the electronic information comprising a 
plurality of pages, the process comprising the steps of: 

displaying a first page of electronic information; 
receiying yerbally delimited keywords; 

conyerting said yerbally delimited keywords into search queries; 

dynamically accessing a database which includes the audio clips based on the search 
queries, wherein the database includes properties that permit audio information to be associated 
with a yisual , wherein the properties include position data indicating the location in the 
electronic information at which the author inserted each audio clip and time data indicating the 
time of recording of each audio clip during a sessio n so that on e audio clip is displayed in 
synchronism with anoth e r audio clip ; 

recording all page nayigations indexed by time so that both the audio clip recorded during 
the session and a sequence of page nayigations can be played back simultaneously: 

playing audio notes associated with said first page; 

displaying a second page of electronic information; and, 

playing audio notes associated with said second page, 

wherein each audio note is associated with a property identifying ^i -the author of said 
audio note. 

84. (Original): The process according to claim 83, further comprising the step of 
receiying user input, 

wherein, in response to said user input, said second page is displayed. 
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85. (Currently Amended): A process of recording audio notes in the form of audio clips 
inserted into an electronic information by an author, the electronic information comprising a 
plurality of pages, the process comprising the steps of: 

displaying a first page of electronic information; 
recording a first set of audio notes; 
receiving verbally delimited keywords; 

converting said verbally delimited keywords into search queries; 

dynamically accessing a database which includes the audio clips based on the search 
queries, wherein the database includes properties that permit audio information to be associated 
with a visual , wherein the properties include position data indicating the location in the 
electronic information at which the author inserted each audio clip and time data indicating the 
time of recording of each audio clip during a sessio n so that on e audio clip is display e d in 
synchronism with anoth e r audio clip ; 

recording all page navigations indexed by time so that both the audio clip recorded during 
the session and a sequence of page navigations can be played back simultaneously; 

associating said first set of audio notes with said first page; 

displaying a second page of electronic information; 

recording a second set of audio notes; and 

associating said second set of audio notes with said second page, 

wherein each set of audio notes is associated with a property identifying an author of said 
set of audio notes. 

86. (Original): The process according to claim 85, further comprising the step of 
receiving user input, 

wherein, in response to said user input, said second page is displayed. 

87. -88. (Canceled) 
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